Information processing apparatus

ABSTRACT

A data processing apparatus is capable of executing a plurality of signal processes. The data processing apparatus switches processes of a pre-processing portion, a data processing portion, and a post-processing portion with a control signal supplied from a function controlling portion corresponding to a command supplied from the outside. Thus, the data processing apparatus executes for example processes for increasing the resolution, generating a picture dedicated for a right eye and a picture dedicated for a left-eye, generating a luminance signal and color difference signals, changing the aspect ratio, generating pictures having difference resolutions, and converting the frame rate for input data corresponding to a request and outputs picture data generated as the processed result to an external device (for example, a displaying device and a record and reproduction device).

This is a divisional of application serial number 09/980,130, filed Nov.28, 2001 now U.S. Pat. No. 7,174,051, which was filed under 35 USC 371and is based upon International Application No. PCT/JP01/02626, filedMar. 29, 2001, with a claim of priority to Japanese Patent ApplicationNo. 2000-093898, filed Mar. 30, 2000, the entirety of which areincorporated herein by reference.

TECHNICAL FIELD

The present invention relates to an information processing apparatus, inparticular, to an information processing apparatus that can execute aplurality of signal processes.

BACKGROUND ART

A signal processing device disposed in a signal processing apparatus isnormally structured so that one function is accomplished.

In other words, to accomplish a plurality of different signal processes,it is necessary to provide devices corresponding to the number ofrequired processes. Thus, for example, it was difficult to compactlystructure and reduce the cost of the apparatus.

DISCLOSURE OF THE INVENTION

The present invention was made from the above-point of view. Accordingto the present invention, corresponding to a predetermined functionexecution request, the internal structure and process contents of thesame hardware are changed so as to execute a plurality of processes.

An information processing apparatus of the present invention, comprisinga plurality of classifying adaptive processing circuits for performing aclassifying adaptive process for input information signals, and aswitching circuit for switching a connection relation among theplurality of classifying adaptive processing circuits.

According to the information processing apparatus of the presentinvention, a plurality of classifying adaptive processing circuitsperform a classifying adaptive process for a desired functioncorresponding to an input information signal. The relation of theconnections of the plurality of classifying adaptive processing circuitsis switched.

At least one of the classifying adaptive processing circuits isconfigured for switching the corresponding classifying adaptive processfor the corresponding information signal as the connection relation ofthe switching circuit is switched.

At least one of the plurality of classifying adaptive processingcircuits is configured for switching the structure of the correspondingclassifying adaptive process as the connection relation of the switchingcircuit is switched.

The structure represents a structure of class taps or a structure ofpredictive taps.

At least one of the plurality of classifying adaptive processingcircuits is configured for switching a coefficient of the correspondingclassifying adaptive process so as to switch the process for thecorresponding information signal as the connection relation is switchedby the switching circuit.

The input information signals are output through the plurality ofclassifying adaptive processing circuits.

A pre-processing circuit is configured for switching a predeterminedprocess as the connection relation is switched. The pre-processingcircuit is configured for performing a predetermined process for thecorresponding input information signal and inputting the processedresult to the classifying adaptive processing circuit.

A post-processing circuit switches a process as the relation of theconnections is switched. The post-processing circuit performs apredetermined process for an output of a classifying adaptive processingcircuit.

Examples of processes that the classifying adaptive processing circuitsaccomplish are as follows.

The information signals are picture data composed of pixel information.One of the plurality of classifying adaptive processing circuits isconfigured for performing the classifying adaptive process correspondingto the pixel information of the corresponding input information signaland predicting pixel information that has to be present between thepixel information of the input information signal and pixel informationadjacent thereto so as to improve the resolution of the picture data.

The information signals are picture data composed of pixel information.One of the plurality of classifying adaptive process circuits isconfigured for performing the classifying adaptive process for thecorresponding input information signal using a prepared left eyecoefficient and predicting pixel information of left-eye picture dataand for performing the classifying adaptive process for thecorresponding input information signal using a prepared right-eyecoefficient and predicting pixel information of right-eye picture dataso as to generate stereo picture data with the left-eye picture data andthe right-eye picture data.

The information signals are picture data composed of pixel information.One of the plurality of classifying adaptive processing circuits isconfigured for performing the classifying adaptive process for thecorresponding input information signal using a prepared luminance signalcoefficient and predicting a luminance signal component of the picturedata and another one of the plurality of classifying adaptive processingcircuits is configured for performing the classifying adaptive processusing prepared color difference signal coefficients and predicting colordifference components of the picture data so as to separate the picturedata into the luminance component and the color difference components.

The information signals are picture data composed of pixel information.At least two of the plurality of classifying adaptive processingcircuits are configured for performing the classifying adaptive processfor the pixel information having different phases and changing thenumber of pixel information that composes the picture data.

The information signals are picture data composed of pixel information.At least two of the plurality of classifying adaptive processingcircuits are configured for performing the classifying adaptive processand obtaining a plurality of picture data having different resolutionscorresponding to the classifying adaptive process performed by theplurality of classifying adaptive processing circuits.

One of the plurality of classifying adaptive processing circuits isconfigured for performing the classifying adaptive process for thecorresponding input information signal and obtaining picture data havinga first resolution and another one of the plurality of classifyingadaptive processing circuits is configured for performing theclassifying adaptive process for picture data having the firstresolution and obtaining picture data having a second resolution.

The information signals are picture data composed of pixel informationand structured in the unit of a frame. One of the plurality ofclassifying adaptive processing circuits is configured for performingthe classifying adaptive process for the corresponding informationsignal that is input in the unit of a frame and generating picture dataof frames chronologically preceded and followed by a frame of the inputinformation signal.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing an example of a basic structure of adata processing apparatus according to the present invention;

FIG. 2 is a block diagram showing an example of the structure of apre-processing portion of the data processing apparatus;

FIGS. 3A and 3B are block diagrams showing an example of the structureof a data processing portion of the data processing apparatus;

FIG. 4 is a block diagram showing an example of the structure of apost-processing portion of the data processing apparatus;

FIG. 5 is a block diagram showing an example of the structure of a dataprocessing apparatus 1 that executes a process for increasing theresolution in process mode (1);

FIG. 6 is a schematic diagram showing pixel arrangements of an SDpicture and an HD picture;

FIGS. 7A and 7B are block diagrams showing an example of the connectionsof the data processing portion in the process mode (1);

FIG. 8 is a block diagram showing an example of the structure of aclassifying adaptive processing portion that performs a predictionprocess;

FIG. 9 is a schematic diagram showing an example of class tap extractioninformation in the process mode (1);

FIG. 10 is a schematic diagram showing other pixel arrangements of an SDpicture and an HD picture;

FIG. 11 is a block diagram showing an example of the structure of alearning device that performs a learning process for calculatingpredictive coefficients that are pre-stored in a coefficient memory of aclassifying adaptive processing portion;

FIG. 12 is a block diagram showing an example of the structure of a dataprocessing apparatus that performs another processing method for theprocess mode (1);

FIGS. 13A and 13B are block diagrams showing an example of theconnections of the data processing portion that performs the otherprocessing method for the process mode (1);

FIG. 14 is a schematic diagram showing an example of class tapextraction information in the other processing method for the processmode (1);

FIG. 15 is a schematic diagram showing a pixel arrangement in the otherprocessing method for the process mode (1);

FIG. 16 is a block diagram showing an example of the structure of thedata processing apparatus that executes a process for generating apicture dedicated for a left eye and a picture dedicated for a right eyein process mode (2);

FIGS. 17A and 17B are block diagrams showing an example of theconnections of the data processing portion in the process mode (2);

FIG. 18 is a schematic diagram showing an example of class tapextraction information in the process mode (2);

FIG. 19 is a schematic diagram showing in more reality predictive tapsin the process mode (2);

FIG. 20 is a schematic diagram for explaining a picture that is learnt;

FIG. 21 is a block diagram showing an example of the structure of theclassifying adaptive processing portion that executes a learning processin the process mode (2);

FIG. 22 is a block diagram showing an example of the structure of thedata processing apparatus that executes a process for generating aluminance signal and color difference signals in process mode (3);

FIGS. 23A and 23B are block diagrams showing an example of theconnections of the data processing portion in the process mode (3);

FIG. 24 is a block diagram showing an example of the structure of thedata processing apparatus that accomplishes a Y/C separating device thatseparates a composite signal NTSC into a component signal (Y, U, V)corresponding to the classifying adaptive process in the process mode(3);

FIG. 25 is a schematic diagram showing an example of class tapextraction information in the process mode (3);

FIG. 26 is a schematic diagram showing an example of the predictive tapextraction information in the process mode (3);

FIG. 27 is a block diagram showing an example of the structure of theclassifying adaptive processing portion that executes a learning processin the process (3) is executed;

FIG. 28 is a block diagram showing an example of the structure of thedata processing apparatus that converts the aspect ratio in process mode(4);

FIGS. 29A and 29B are block diagrams showing an example of theconnections of the data processing portion in the process mode (4);

FIG. 30 is a schematic diagram for explaining the classifying adaptiveprocessing portion of the input data processing portion that performs apredictive process in the process mode (4);

FIG. 31 is a block diagram showing an example of the structure of thedata processing apparatus that generates a picture having a differentresolution in process mode (5);

FIGS. 32A and 32B are block diagrams showing an example of theconnections of the data processing portion 12 in the process mode (5);

FIG. 33 is a block diagram showing an example of the structure of thedata processing apparatus that converts a frame rate in process mode(6);

FIGS. 34A and 34B are block diagrams showing an example of theconnections of the data processing portion in the process mode (6);

FIG. 35 is a schematic diagram for explaining frame structures of aninput picture and an output picture; and

FIG. 36 is a schematic diagram showing an example of class tapextraction information in the process mode (6).

BEST MODES FOR CARRYING OUT THE INVENTION

Next, with reference to the accompanying drawings, an embodiment of thepresent invention will be described. FIG. 1 shows an outline of anexample of the structure of a data processing apparatus 1 according tothe present invention. As shown in FIG. 1, a picture data as input datais supplied from a picture processing device such as a televisionreceiver, a video camera, a VTR, a scanner, or a digital camera to thedata processing apparatus 1 according to the present invention.

The data processing apparatus 1 according to the embodiment can executeprocess modes that accomplish for example the following contentscorresponding to requests. Picture data that is generated as theprocessed result of each process mode is output as output data to anexternal device (for example, a displaying device or a recording andreproducing device).

(1) Increase of resolution (referred to as process mode (1)),

(2) Generation of a picture dedicated for right eye and a picturededicated for left eye (referred to as process mode (2)),

(3) Generation of a luminance signal and color difference signals(referred to as process mode (3)),

(4) Change of aspect ratio (referred to as process mode (4)),

(5) Generation of a picture having a different resolution.(referred toas process mode (5)), and

(6) Conversion of frame rate (referred to as process mode (6)).

The details of the processes of those process modes performed by thedata processing apparatus 1 will be described later.

The data processing apparatus 1 has a function controlling portion 11, adata processing portion 12, a pre-processing portion 13, and apost-processing portion. 14. A command that causes a desired process tobe executed is input to the function controlling portion 11. Thefunction controlling portion 11 supplies a control signal correspondingto the input command to the data processing portion 12 so as toinitialize the data processing portion 12 and executes a processcorresponding to the command. In addition, the control signal that isoutput from the function controlling portion 11 is supplied to thepre-processing portion 13 and the post-processing portion 14. Thecontrol signal causes the pre-processing portion 13 and thepost-processing portion 14 to execute predetermined processes.

The pre-processing portion 13 performs a predetermined pre-process forpicture data as input data corresponding to the control signal suppliedfrom the function controlling portion 11 and supplies the processed datato the data processing portion 12. In addition, the control signal issupplied from the function controlling portion 11 to the data processingportion 12. The data processing portion 12 is initialized with thecontrol signal. The data processing portion 12 performs a classifyingadaptive process corresponding to a process designated with the controlsignal for the supplied input data and outputs data generated as theprocessed result. An output of the data processing portion 12 issupplied to the post-processing portion 14. The post-processing portion14 performs a predetermined post-process for the output of the dataprocessing portion 12 corresponding to the control signal supplied fromthe function controlling portion 11 and outputs the result as outputdata to an external device.

FIG. 2 shows an example of the structure of the pre-processing portion13. The pre-processing portion 13 has selectors 130 and 132 and aplurality of delaying circuits 131A, 132B, and so forth. A controlsignal that is supplied from the function controlling portion 11 isinput to the selectors 130 and 132 so as to control the selections ofthe signal paths of the selectors 130 and 132. The delaying circuits131A, 131B, and so forth have different delay times. For example, thedelaying circuit 131A delays an input signal in the unit of a pixel. Thedelaying circuit 131B delays an input signal in the unit of a line. Thedelaying circuit 131C delays an input signal in the unit of a frame.Signals that are output from the delaying circuits 131A, 131B, 131C, andso forth are input to the selector 132. The selector 132 selects thesignals that are output from the delaying circuits 131A, 131B, 131C, andso forth corresponding to the control signal supplied from the functioncontrolling portion 11. The signals selected by the selector 132 areoutput from the pre-processing portion 13.

Alternatively, in the pre-processing portion 13, the signals that areoutput from the selector 130 may be directly input to the selector 132not through the delaying circuits 131A, 131B, 131C, and so forth.Further alternatively, the selector 130 may output the same signal inparallel as a plurality of outputs. Further alternatively, thepre-processing portion 13 may have a processing circuit other than thedelaying circuits 131A, 131B, and so forth.

FIG. 3 shows an example of the structure of the data processing portion12. The data processing portion 12 is composed of an input dataprocessing portion 21, a selector 22, an intermediate data processingportion 23, an output data processing portion 24, and a selector 25.

The input data processing portion 21 has a plurality of classifyingadaptive processing portions (in this example, four classifying adaptiveprocessing portions 31-1 to 31-4 (hereinafter simply referred to asclassifying adaptive processing portion 31 when it is not necessary todistinguish them) (such notation applies to other structural portions).The classifying adaptive processing portion 31 executes a classifyingadaptive process for supplied data (namely, input data that is input tothe data processing apparatus 1 or data that is input to the dataprocessing apparatus 1 and pre-processed by the pre-processing portion13) and supplies the generated data as the processed result to theselector 22.

The data that is processed (generated) in the input data processingportion 21 and the intermediate data processing portion 23 is suppliedto the selector 22. The select portion 22 selects a destination for thesupplied data (the destination is for example a memory 41 of theintermediate data processing portion 23 or a classifying adaptiveprocessing portion 51 of the output data processing portion 24) andsupplies the data to the selected destination.

The intermediate data processing portion 23 has a plurality of memories(in this example, four memories 41-1 to 41-4). The intermediate dataprocessing portion 23 performs a write control and a read control of thememory 41 for data supplied from the selector 22. For example, theintermediate data processing portion 23 executes a process for changinga horizontal scanning sequence to a vertical scanning sequence. Datathat is read from the memory 41 of the intermediate data processingportion 23 is returned to the selector 22.

The output data processing portion 24 has a plurality of classifyingadaptive processing portions (in this example, eight classifyingadaptive processing portions 51-1 to 51-8). The classifying adaptiveprocessing portion 51 executes a classifying adaptive process for thedata supplied from the selector 22 corresponding to the control signalreceived from the function controlling portion 11 and outputs thegenerated data as the processed result. Data that is output from theclassifying adaptive processing portions 51-1 to 51-8 is input to theselector 25. The selector 25 selects data corresponding to the controlsignal that is supplied from the function controlling portion 11 andoutputs the selected data.

FIG. 4 shows an example of the structure of the post-processing portion14. The post-processing portion 14 has a selector 140, a plurality ofprocessing portions 141A, 141B, and so forth, and a selector 142. Datathat is output from the data processing portion 12 is input to theselector 140. The selector 140 switches paths of the input datacorresponding to a control signal that is supplied from the functioncontrolling portion 11 and inputs the switched data to the plurality ofthe processing portions 141A, 141B, and so forth.

According to the embodiment, the processing portion 141A is a linesequence converting portion that performs a conversion in the unit of aline. The processing portion 141B is a scanning direction convertingportion that performs a conversion in the unit of a pixel. Theprocessing portion 141C is a multiplexing portion that multiplexes data.The processing portions 141A, 141B, and so forth process input datacorresponding to a control signal supplied from the function controllingportion 11.

Next, the structure and operation of the data processing apparatus 1that executes the processes for accomplishing the above-describedprocess modes (1) to (6) will be described.

FIG. 5 shows an example of the structure of the data processingapparatus 1 that executes the process for increasing the resolution inthe process mode (1). In this example, an SD (Standard Density) picture(for example, picture data in an interlace format of which the number oflines is 525) (hereinafter, sometimes referred to as SD picture data) isinput as input data from an external device to the data processingapparatus 1. It is assumed that the data processing apparatus 1generates an HD (High Density) picture (for example, picture data in aprogressive format of which the number of lines is 525 and of which thenumber of pixels in the horizontal direction is twice that of the SDpicture) (hereinafter, sometimes, the HD picture data is referred to asHD picture data).

In this example, a command that causes the process for converting an SDpicture into an HD picture to be executed is additionally input to thefunction controlling portion 11. The function controlling portion 11outputs a control signal corresponding to the input command to the dataprocessing portion 12.

The data processing portion 12 generates data of each line of the HDpicture with the SD picture that is input as the input data to the dataprocessing apparatus 1 corresponding to the control signal received fromthe function controlling portion 11. Accurately speaking, the dataprocessing portion 12 predicts pixels arranged on each line.

In reality, as shown in FIG. 6, when a predetermined field of an SDpicture is composed of lines L_(SD-1), L_(SD-2), L_(SD-3), and so forthon which pixels denoted by large circles ◯ (hereinafter referred to asSD pixels) are arranged, the data processing portion 12 generates linesL_(HD-1), L_(HD-2), L_(HD-3) on which pixels of an HD picture denoted bysmall circles ∘ are arranged at the same positions as the linesL_(SD-1), L_(SD-2), L_(SD-3), and so forth of the SD picture(hereinafter, sometimes, a line of an HD picture at the same position asa line of an SD picture is referred to as line A) and lines L_(HD-2),L_(525P-4), and so forth of the HD picture at positions that are notsame as the lines of the SD picture (hereinafter, a line of an HDpicture at a position that is not same as a line of an SD picture isreferred to as line B).

The data processing portion 12 separately generates pixels of line A andpixels of line B. In the post-processing portion 14 disposed downstreamof the data processing portion 12, the selector 140 selects the linesequence converting portion 141A and outputs line A and line B suppliedfrom the data processing portion 12 in line sequence. In other words, HDpicture data is output as output data from the line sequence convertingportion 141A to an external device through the selector 142.

In other words, in this example, the data processing apparatus 1non-interlaces an SD picture as an interlace format picture, doubles thenumber of pixels in the horizontal direction, and thereby generates anHD picture having pixels four times larger than an SD pixel.

FIG. 7 shows an example of the connections of the data processingportion 12 that executes the process for increasing the resolution inthe process mode (1). In this example, in the input data processingportion 21, the two classifying adaptive processing portions 31-1 and31-2 are used. SD picture data that is input to the data processingapparatus 1 is supplied as input data to the classifying adaptiveprocessing portions 31-1 and 31-2.

The classifying adaptive processing portion 31-1 executes a classifyingadaptive process for generating line A (in the example shown in FIG. 6,L_(HD-1), L_(HD-3), LD_(HD-5), and so forth) that compose an HD picturewith supplied SD picture data. In other words, in the classifyingadaptive process, a process for predicting HD pixels arranged on line Ais performed. The classifying adaptive processing portion 31-2 executesa classifying adaptive process for generating line B (in the exampleshown in FIG. 6, L_(HD-2), L_(HD-4), and so forth) that compose an HDpicture with an SD picture. In other words, in the classifying adaptiveprocess, a process for predicting HD pixels arranged on line B isperformed.

Pixel data of line A and line B generated by the classifying adaptiveprocessing portions 31-1 and 31-2 is supplied to the select portion 22.

The select portion 22 supplies pixel data of line A and line Bsimultaneously supplied from the classifying adaptive processingportions 31-1 and 31-2 of the input data processing portion 21 to thepost-processing portion 14. In the post-processing portion 14, theselector 140 selects the line sequence converting portion 141Acorresponding to a control signal supplied from the function controllingportion 11. Thus, line A and line B are supplied to the line sequenceportion 141A. Line A and line B that compose the HD picture that aresimultaneously supplied to the line sequence converting portion 13 areconverted into a progressively scanned picture in the line sequence andoutput to an external device.

The intermediate data processing portion 23 and the output dataprocessing portion 24 are not used in the process mode.

FIG. 8 shows an example of the structure of the classifying adaptiveprocessing portion 31-1 of the input data processing portion 21. As wasdescribed above, the classifying adaptive processing portion 31-1performs the process for line A. The classifying adaptive processingportion 31-1 comprises a classifying portion 61, an adaptive processingportion 62, and a register group 63. The classifying portion 61, theadaptive processing portion 62, and the register group 63 select a bankfor the process mode corresponding to a control signal that is outputfrom the function controlling portion 11 to the data processing portion12 and execute various processes corresponding to a coefficient set ofthe selected bank.

The classifying portion 61 is composed of a class tap extracting circuit71 and a classifying circuit 72. Successively designating HD pixels online A of an HD picture as considered pixels, the classifying portion 61classifies the considered pixels as predetermined classes.

In other words, the class tap extracting circuit 71 selects SD pixels(hereinafter, sometimes referred to as class taps) for each consideredpixel from an SD picture, extracts the class taps, and supplies them tothe classifying circuit 72.

A register 63A of the register group 63 registers information(hereinafter referred to as class tap extraction information) of aselected pattern of SD pixels as class taps for each classifyingadaptive process executed by the classifying adaptive processing portion31-1. In other words, before extracting class taps, the class tapextracting circuit 71 reads class tap extraction information necessaryin the example (the class tap extraction information corresponds to acontrol signal supplied from the function controlling portion 11 to thedata processing portion 12) and extracts class taps corresponding to theclass tap extraction information.

FIG. 9 shows an example of class tap extraction information in theprocess mode (1). In FIG. 9, ● (black circle) and ◯ (white circle)represent SD pixels. Assuming that a considered pixel is a black circle,class taps are composed of 3 pixels×3 pixels that are arranged aroundthe considered pixel and that include the considered pixel. Predictivetaps (that will be described later) are composed of 5 pixels×5 pixelswider than class taps by one pixel. In FIG. 9, X marks represent HDpixels that are finally formed corresponding to a considered pixel. Asrepresented in area A shown in FIG. 9, four HD pixels are formed withone considered pixel. As a result, above-described line B is formed.

Next, with reference to FIG. 10, the process will be described in moredetail. FIG. 10 is a schematic diagram showing the detail of onesequence containing a considered pixel in the vertical direction in FIG.9. For example, SD pixels denoted by large circles are arranged in onesequence containing a considered pixel of an SD picture of apredetermined field (namely, in the vertical direction). On the otherhand, it is assumed that HD pixels denoted by small circles are arrangedin one sequence corresponding to the above-described considered pixel ofthe SD picture. When HD pixel Y₁ of the SD picture is a consideredpixel, corresponding to class tap extraction information that is readfrom the register 63A, a total of three SD pixels that are SD pixel X₂that is the closest to HD pixel Y₁ (at the same position in the exampleshown in FIG. 10) and two SD pixels X₁ and X₃ vertically adjacent to SDpixel X₂ are selected as class taps of the considered pixel (HD pixel)Y₁.

Returning to FIG. 8, the classifying circuit 72 of the classifyingportion 61 detects a feature of SD pixels that compose class tapssupplied from the class tap extracting circuit 71 (for example, thedistribution of pixel values) and supplies a class code that has beenassigned for each feature to the adaptive processing portion 62(coefficient memory 83). The class code is supplied as an address to thecoefficient memory 83. As a pixel value, for example a luminance valueof a pixel can be used.

Since each pixel is assigned eight-bit data, in the example, assumingthat each SD pixel is assigned eight-bit data, the number of classesbecomes huge. Thus, the required memory capacity increases.

Thus, actually, when classifying each considered pixel, the classifyingcircuit 72 performs the process for decreasing (compressing) the numberof bits of each pixel that composes class taps. As an example of themethod for the compressing process for the number of bits, ADRC(Adaptive Dynamic Range Coding) process is known.

In the ADRC, the maximum pixel value MAX and the minimum pixel value MINare detected from pixels that compose a process block (class taps). Inaddition, the difference DR between the pixel value MAX and the pixelvalue MIN (=pixel value MAX−pixel value MIN) is calculated. The obtainedDR is the dynamic range DR of the process block. The pixel value MIN issubtracted from each pixel value that composes the process block. Eachsubtracted value is divided by DR/2^(K). As a result, each pixel valuethat composes class taps as-the process block is re-quantized to K bitsthat are smaller than the originally assigned bits (eight bits). Forexample, when K=1, in the example shown in FIG. 10, the number ofpatterns of pixel values of three SD pixels is (2¹)³. In comparison withthe case that the ADRC is not performed, the number of patterns can bedecreased.

The compressing process of the classifying circuit 72 is not limited tothe ADRC. Instead, another compressing process such as vectorquantization may be used.

The classifying circuit 72 decides the class of the considered pixelcorresponding to the K-bit pixel value of each SD pixel that composesthe class taps.

Before classifying the considered pixel, the classifying circuit 72reads class tap extraction information necessary in the process modefrom the register 63A of the register group 63, extracts the class tapscorresponding to the class tap extraction information, and classifiesthe considered pixel with the extracted class taps.

The adaptive processing portion 62 is composed of a predictive tapextracting circuit 81, a predictive calculating circuit 82, and acoefficient memory 83. The predictive tap extracting circuit 81 selectsSD pixels of the SD picture supplied to the adaptive processing portion62 as predictive taps and supplies the predictive taps to the predictivecalculating circuit 82.

A register 63B of the register group 63 registers information ofselected patterns of predictive taps of the SD picture (hereinafter,sometimes, referred to as predictive tap extraction information) foreach classifying adaptive process executed by the classifying adaptiveprocessing portion 31-1. In other words, before forming predictive taps,the predictive tap extracting circuit 81 reads required predictive tapextraction information (predictive tap extraction informationcorresponding to a control signal supplied from the function controllingportion 11 to the data processing portion 12) from the register 63B andextracts the predictive taps from the input data corresponding to thepredictive tap extraction information. In the process mode (1), as wasdescribed with reference to FIG. 9, the predictive taps are composed of5 pixels×5 pixels arranged around the considered pixel.

The coefficient memory 83 stores predictive coefficient sets (that areobtained in the learning process that will be described later) forindividual classes. When a class is supplied from the classifyingcircuit 72 of the classifying portion 61 a predictive coefficient set isread from the address corresponding to the class and supplied to thepredictive calculating circuit 82.

The coefficient memory 83 is composed of a plurality of banks. Each bankstores predictive coefficient sets corresponding to the individualprocess modes. Each bank stores predictive coefficient sets ofcorresponding types. A register 63C of the register group 63 registersbank selection information that is information of bank selectionscorresponding to process modes.

Before reading a predictive coefficient set, the coefficient memory 83reads bank selection information (that is coefficient informationcorresponding to a control signal supplied from the function controllingportion 11 to the data processing portion 12) corresponding to theprocess mode and sets a bank corresponding to the bank selectioninformation. As described above, in the example, since the classifyingadaptive processing portion 31-1 performs the process for line A, thecoefficient memory 83 sets a bank that stores a predictive coefficientset corresponding to line A. Predictive coefficients are read from anaddress of the bank corresponding to the supplied class and supplied tothe predictive calculating circuit 82.

Predictive taps are supplied from the predictive tap extracting circuit81 to the predictive calculating circuit 82. In addition, a predictivecoefficient set is supplied from the coefficient memory 83 to thepredictive calculating circuit 82.

The predictive calculating circuit 82 performs a calculation ofExpression (1) that is a linear combination model defined by a linearcombination of a predictive coefficient W and a pixel value x using apredictive coefficient set (predictive coefficients w₁, w₂, and soforth) supplied from the coefficient memory 83 and pixel values (pixelvalues x₁, x₂, and so forth) of pixels that compose predictive tapssupplied from the coefficient memory 83, obtains a predictive value E[y] of the considered pixel (HD pixel) y, and treats the calculatedresult as the pixel value of a HD pixel (HD pixel arranged on line A).E [y]=w ₁ x ₁ +w ₂ x ₂+ . . .   (1)

Alternatively, using a non-linear combination model, a calculation canbe performed.

As described above, pixel values of individual HD pixels arranged online A of an HD picture are predicted. Thus, line A is generated.

The structure of which the classifying adaptive processing portion 31-2performs a predictive process as a classifying adaptive process in theprocess mode (1) is basically the same as the structure of theabove-described classifying adaptive processing portion 31-1. In otherwords, in the classifying adaptive processing portion 31-2, the classtap extracting circuit 71 and the coefficient memory 83 of theclassifying portion 61 and the adaptive processing portion 62 extractclass taps, predictive taps, and a predictive coefficient setcorresponding to required tap extraction information, predictive tapextraction information, and bank selection information. As a result,pixel values of HD pixels arranged on line B of an HD picture (namely, aline at a position that is not the same as a line of the SD picture) arepredicted. For example, the pixel value of HD pixel Y₂ shown in FIG. 10is predicted. As a result, line B is generated.

FIG. 11 shows an example of the structure of a learning device thatperforms a learning process for calculating predictive coefficients thatare pre-stored to the coefficient memory 83 shown in FIG. 8. In theprocess mode (1), the learning process is performed for line A and lineB. The learnt results for line A and line B are separately output.

An HD picture as teacher data in the learning process is supplied toboth a thin-out circuit 91 and a teacher data extracting circuit 95. Thethin-out circuit 91 thins out pixels of the HD picture as the teacherdata so as to generate an SD picture. The thin-out circuit 91 suppliesthe SD picture to a classifying portion 92 and a predictive tapextracting circuit 93. For example, the thin-out circuit 91 halves thenumber of pixels in each of the horizontal direction and the verticaldirection of the HD picture so as to form an SD picture.

The classifying portion 92 decides the class of the considered pixel inthe same process as the classifying portion 61 shown in FIG. 8 andsupplies the decided class to address terminals (AD) of a predictive tapmemory 94 and a teacher data memory 96. The predictive tap extractingcircuit 93 performs the same process as the predictive tap extractingcircuit 81 shown in FIG. 8 so as to extract predictive taps. Thepredictive tap extraction circuit 93 supplies the extracted predictivetaps to the predictive tap memory 94.

The predictive tap memory 94 stores the predictive taps supplied fromthe predictive tap extracting circuit 93 at an address corresponding tothe class supplied from the classifying portion 92.

On the other hand, the teacher data extracting circuit 95 extracts an HDpixel as a considered pixel (designated by the classifying portion 92and the predictive tap extracting circuit 93) from the supplied HDpicture and supplies the HD pixel as teacher data to the teacher datamemory 96.

The teacher data memory 96 stores teacher data supplied from the teacherdata extracting circuit 95 at an address corresponding to the classsupplied from the classifying portion 92.

The above-described process is performed for all pixels as consideredpixels that compose an HD picture prepared for the learning process.

As a result, at the same address of the teacher data memory 96 or thepredictive tap memory 94, an HD pixel of the class corresponding to theaddress as teacher data and SD pixels at the positions of predictivetaps of the HD pixels as learnt data are stored.

In the predictive tap memory 94 and the teacher data memory 96, aplurality of pieces of information can be stored at the same address.Thus, a plurality of pieces of learnt data and teacher data that areclassified as the same class can be stored at the same address.

A calculating circuit 97 reads predictive taps as learnt data or HDpixels as teacher data stored at the same address of the predictive tapmemory 94 or the teacher data memory 96, respectively. Using thepredictive taps or HD pixels that are read, by for example the method ofleast squares, a predictive coefficient of which the difference betweenpredictive value and teacher data becomes minimum is obtained. In otherwords, the calculating circuit 97 solves a normal equation as Expression(2) for each class so as to obtain a predictive coefficient.

$\begin{matrix}\left( \begin{matrix}{{{\left( {\sum\limits_{i = 1}^{I}{x_{i\; 1}x_{i\; 1}}} \right)w_{1}} + {\left( {\sum\limits_{i = 1}^{I}{x_{i\; 1}x_{i\; 2}}} \right)w_{2}} + \ldots + {\left( {\sum\limits_{i = 1}^{I}{x_{i\; 1}x_{ij}}} \right)w_{j}}} = \left( {\sum\limits_{i = 1}^{I}{x_{i\; 1}y_{i}}} \right)} \\{{{\left( {\sum\limits_{i = 1}^{I}{x_{i\; 2}x_{i\; 1}}} \right)w_{1}} + {\left( {\sum\limits_{i = 1}^{I}{x_{12}x_{i\; 2}}} \right)w_{2}} + \ldots + {\left( {\sum\limits_{i = 1}^{I}{x_{i\; 2}x_{ij}}} \right)w_{j}}} = \left( {\sum\limits_{i = 1}^{I}{x_{12}y_{i}}} \right)} \\\cdots \\{{{\left( {\sum\limits_{i = 1}^{I}{x_{ij}x_{i\; 1}}} \right)w_{1}} + {\left( {\sum\limits_{i = 1}^{I}{x_{ij}x_{i\; 2}}} \right)w_{2}} + \ldots + {\left( {\sum\limits_{i = 1}^{I}{x_{ij}x_{ij}}} \right)w_{j}}} = \left( {\sum\limits_{i = 1}^{I}{x_{ij}y_{i}}} \right)}\end{matrix} \right. & (2)\end{matrix}$

The predictive coefficient set that has been obtained in theabove-described manner is stored to the coefficient memory 83 shown inFIG. 8.

Since the structure of the classifying adaptive processing portion 31-2of the input data processing portion 21 that performs the learningprocess is the same as that of the above-described classifying adaptiveprocessing portion 31-1, the description will be omitted.

In the above-described example, luminance values are used as pixelvalues. When a signal is composed of luminance and color differences,the above-described process is performed for each of the luminance andcolor differences so as to generate an HD picture. In the case, theclassifying adaptive processing portions 31-3 and 31-4 of the input dataprocessing portion 21 may execute a classifying adaptive process usingcolor differences and generate line A and line B that compose an HDpicture.

The classifying adaptive process for converting an interlaced pictureinto a non-interlaced picture is described in Japanese PatentApplication No. HEI 10-208116 (Japanese Patent Laid-Open Publication No.2000-41223 laid open on Feb. 8, 2000) and the corresponding U.S. patentapplication (Ser. No. 09/358,272, filed on Jul. 21, 1999) both of whichwere filed by the applicant of the present invention.

Although the normal equation as Expression (2) used in obtainingpredictive coefficients has been disclosed in prior art (for example,Japanese Patent Laid-Open Publication No. 2000-41223 laid open on Feb.8, 2000), for allowing the reader to easily understand the presentapplication, it will be described in detail.

To generalize Expression (1) of the linear combination model defined bylinear combination of predictive coefficients and pixel values forcalculating a predictive value E, defining a matrix W as a set ofpredictive coefficients w_(j), a matrix X as a set of student datax_(ij), and a matrix Y′ as a set of predictive values E [y_(i)]asExpression (3), an observation equation as Expression (4) is satisfied.

$\begin{matrix}{X = \begin{bmatrix}x_{11} & x_{12} & \ldots & x_{1j} \\x_{21} & x_{22} & \ldots & x_{2j} \\\ldots & \ldots & \ldots & \ldots \\x_{I\; 1} & x_{I\; 2} & \ldots & x_{Ij}\end{bmatrix}} & (3) \\{{{W = \begin{bmatrix}w_{1} \\w_{2} \\\ldots \\w_{j}\end{bmatrix}},{Y^{\prime} = \begin{bmatrix}{E\left\lbrack y_{1} \right\rbrack} \\{E\left\lbrack y_{2} \right\rbrack} \\\ldots \\{E\left\lbrack y_{j} \right\rbrack}\end{bmatrix}}}{{XW} = Y^{\prime}}} & (4)\end{matrix}$

In this case, a component x_(ij) of the matrix X represents j-th studentdata of an i-th set of student data (a set of student data used forpredicting i-th teacher data y_(i)). A component w_(j) of the matrix Wrepresents a predictive coefficient that is multiplied by j-th studentdata of a set of the student data. y_(i) represents i-th teacher data.Thus, E [y_(i)] represents a predictive value of i-th teacher data. Inother words, y on the left side of Expression (1) is obtained byomitting the suffix i of the component y_(i) of the matrix Y. x₁, x₂, .. . of the right side of Expression (1) are obtained by omitting thesuffix i of the component x_(ij) of the matrix X.

Now, by applying the method of least squares to the observationequation, a predictive value E [y] close to a pixel value y of an HDpicture is obtained. In this case, when the matrix E as a set ofresiduals e of a predictive value E [y] to a pixel-value y of an HDpixel is defined with Expression (5), a residual formula as Expression(6) is satisfied with Expression (4).

$\begin{matrix}{{E = \begin{bmatrix}e_{1} \\e_{2} \\\ldots \\e_{j}\end{bmatrix}},{Y = \begin{bmatrix}y_{1} \\y_{2} \\\ldots \\y_{j}\end{bmatrix}}} & (5) \\{{XW} = {Y + E}} & (6)\end{matrix}$

In this case, the predictive coefficient w_(i) for obtaining thepredictive value E [y] close to a pixel value y of an HD picture isobtained by minimizing the square error of Expression (7).

$\begin{matrix}{\sum\limits_{i = 1}^{I}e_{i}^{2}} & (7)\end{matrix}$

Thus, when the result of which the square error is differentiated withthe predictive coefficient W_(j) becomes 0, (in other words) thepredictive coefficient w_(j) that satisfies Expression (8) becomes anoptimum value for obtaining the predictive value E [y] that is close tothe pixel value y of an HD pixel.

$\begin{matrix}{{{e_{1}\frac{\partial e_{1}}{\partial W_{j}}} + {e_{2}\frac{\partial e_{2}}{\partial W_{j}}} + \ldots + {e_{1}\frac{\partial e_{I}}{\partial W_{j}}}} = {0\left( {{j = 1},2,\ldots\mspace{11mu},J} \right)}} & (8)\end{matrix}$

Thus, when Expression (6) is differentiated with the predictivecoefficient w_(j), Expression (9) is satisfied.

$\begin{matrix}{{\frac{\partial e_{i}}{\partial W_{1}} = X_{i\; 1}},{\frac{\partial e_{i}}{\partial W_{2}} = X_{i\; 2}},\ldots\mspace{11mu},{\frac{\partial e_{i}}{\partial W_{j}} = {X_{ij}\left( {{i = 1},2,\ldots\;,I} \right)}}} & (9)\end{matrix}$

With Expression (8) and Expression (9), Expression (10) is obtained.

$\begin{matrix}{{{\sum\limits_{i = 1}^{I}{e_{i}x_{i\; 1}}} = 0},{{\sum\limits_{i = 1}^{I}{e_{i}x_{i\; 2}}} = 0},\ldots\mspace{11mu},{{\sum\limits_{i = 1}^{I}{e_{i}x_{i\; 1}}} = 0}} & (10)\end{matrix}$

Considering the relation among the student data x_(ij), predictivecoefficients w_(j), teacher data y_(i), and residuals e_(j) of theresidual equation as Expression (6), with Expression (10), the normalequations of Expression (2) can be obtained. In this case, for eachnormal equation as Expression (2), it is preferred to prepare apredetermined number of sets of student data x_(ij) and teacher datay_(i). Thus, the normal equations corresponding to the number J ofpredictive coefficients w_(j) can be obtained. By solving Expression(2), an optimum predictive coefficient w_(j) can be obtained.

FIG. 12 shows an example of the structure of the data processingapparatus 1 that performs another processing method for increasing theresolution in the process mode (1). In the other processing method forthe process mode (1), the post-processing portion 14 selects a scanningdirection converting portion 14B.

The function controlling portion 11 outputs a control signalcorresponding to a command that is input in this case to the dataprocessing portion 12.

The data processing portion 12 is initialized with a control signalsupplied from the function controlling portion 11. Thereafter, thenumber of pixels in the horizontal direction of an SD picture as inputdata supplied from an external device is doubled. Thereafter, ahorizontal scanning sequence (television raster scanning sequence) isconverted into a vertical scanning sequence. Moreover, the number ofpixels in the vertical direction is doubled. In other words, in thecase, an HD picture with a higher resolution than an SD picture isgenerated.

However, since picture data that is output from the data processingportion 12 is still in the vertical scanning sequence, it is necessaryto restore the picture data to the horizontal scanning sequence. Thus,in the data processing apparatus 1, the post-processing portion 14disposed downstream of the data processing portion 12 selects thescanning direction converting portion 141B corresponding to a controlsignal supplied from the function controlling portion 11.

As with the intermediate data processing portion 23 of the dataprocessing portion 12, the scanning direction converting portion 141Bhas a memory. By controlling writing and/or reading process of thepicture data that is supplied from the data processing portion 12to/from the memory, the vertical scanning sequence is converted into thehorizontal scanning sequence. For example, picture data that is suppliedfrom the data processing portion 12 to the scanning direction convertingportion 141B is address-controlled so that pixels are written in thevertical scanning direction to the memory. When picture data is readfrom the memory, the memory is address-controlled so that pixels areread in the horizontal scanning direction. Thus, the scanning directionof data is converted from the vertical scanning direction to thehorizontal scanning direction. The picture data restored to thehorizontal scanning sequence by the scanning direction convertingportion 141B is output as output data to an external device.

FIG. 13 shows an example of the connections of the data processingportion 12 that performs the other processing method in the process mode(1). In this example, in the input data processing portion 21, oneclassifying adaptive processing portion 31-1 is used. SD picture datathat is input as input data to the data processing apparatus 1 issupplied to the classifying adaptive processing portion 31-1.

The classifying adaptive processing portion 31-1 performs a classifyingadaptive process for generating a picture whose number of pixels in thehorizontal direction is twice as many as the supplied SD picture data(hereinafter, sometimes, the generated picture is referred to asintermediate picture) and supplies picture data of the intermediatepicture (hereinafter, sometimes, the picture data is referred to asintermediate picture data) to the selector 22.

In the example, in the intermediate data processing portion 23, thememory 41-1 is used. The intermediate picture data is supplied from theselect portion 22 to the memory 41-1. The intermediate data processingportion 23 address-controls the memory 41-1 so that the data written inthe horizontal scanning direction to the memory 41-1 is read in thevertical scanning direction therefrom. Thus, the intermediate dataprocessing portion 23 converts the horizontal scanning sequence of theintermediate picture into the vertical scanning sequence and returns theconverted picture to the selector 22.

In this example, in the output data processing portion 24, theclassifying adaptive processing portion 51-1 is used. The intermediatepicture data is supplied from the selector 22 to the classifyingadaptive processing portion 51-1. The classifying adaptive processingportion 51-1 performs a classifying adaptive process for doubling thenumber of pixels in the vertical direction of the intermediate picturecorresponding to the intermediate picture data supplied from the selectportion 22 and supplies the generated picture data to thepost-processing portion 14 disposed downstream of the classifyingadaptive processing portion 51-1.

The picture data supplied to the post-processing portion 14 is suppliedto the scanning direction converting portion 141B selected by theselector 140 corresponding to a control signal supplied from thefunction controlling portion 11. The scanning direction convertingportion 141B performs a write control and a read control for the memoryso that picture data that is written in the vertical scanning directionto the memory is read in the horizontal scanning direction therefrom. Asa result, the vertical scanning sequence of the picture data is restoredto the horizontal scanning sequence. The picture (HD picture) restoredto the horizontal scanning sequence by the scanning direction convertingportion 141B is output as output data to an external device.

Next, the predictive process of the classifying adaptive processingportion 31-1 for the input data processing portion 21 according to theother processing method in the process mode (1) will be described.

The class tap extracting circuit 71 (see FIG. 8) of the classifyingadaptive processing portion 31-1 reads class tap extraction informationrequired in this example from the register 63A of the register group 63,forms class taps corresponding to the class tap extraction information,and supplies the class taps to the classifying circuit 72.

FIG. 14 shows an example of class tap extraction informationcorresponding to the other processing method in the process mode (1).FIG. 14 shows an example of class tap extraction information in theprocess for doubling the number of pixels in the horizontal scanningdirection in the process mode (1). In FIG. 14, ● (black circle) and ◯(white circle) represent SD pixels. Assuming that a considered pixel isa black circle pixel, class taps are composed of seven pixels arrangedon a line of which the considered pixel is placed at the center thereof.On the other hand, predictive taps that will be described later arecomposed of 11 pixels that are wider than the class taps by two pixels.

The class taps and predictive taps used for the process for doubling thenumber of pixels in the vertical direction may be arranged in thevertical direction rather than the horizontal direction shown in FIG.14.

Next, with reference to FIG. 15, this process will be described indetail. It is assumed that lines K−1 to K+1 are arranged in apredetermined field of an SD picture as input data. In FIG. 15, largecircles ◯ represent SD pixels. Small circles represent pixels of an HDpicture whose number of pixels in the horizontal direction is twice asmany as an SD picture. The pixels are arranged as input data on linesK−1 to K+1 of a predetermined field of the SD picture as shown in FIG.15.

Now, the case that pixel Y_(K,5) of an intermediate picture on line K isa considered pixel is considered. In the example, according to the classtap extraction information that is read from the register group 63A, atotal of seven SD pixels that are SD pixel X_(K,3) that is predicted asa pixel having the highest correlation to pixel Y_(K,5) on line K, twoSD pixels X_(K,2) and X_(K,1) arranged on the left of SD pixel X_(K,3),two SD pixels X_(K,4) and X_(K,5) arranged on the right of SD pixelX_(K,3), and SD pixels X_(k−1,3) and SD pixel X_(K+1,3) arranged on lineK−1 and line K+1 at the positions corresponding to SD pixel X_(K,3) areselected as class taps.

Before classifying the considered pixel, the classifying circuit 72reads class tap extraction information required in the process mode fromthe register 63A of the register group 63, extracts class tapscorresponding to the class tap extraction information, and classifiesthe considered pixel using the extracted class taps. The decided classof the considered pixel is supplied as an address to the coefficientmemory 83 of the adaptive processing portion 62.

The predictive tap extracting circuit 81 of the adaptive processingportion 62 reads predictive tap extraction information required in theexample from the register 63B of the register group 63, forms predictivetaps corresponding to the predictive tap extraction information, andsupplies the predictive taps to the predictive calculating circuit 82.

The coefficient memory 83 reads coefficient information required in theexample from the register 63C of the register group 63. In thecoefficient memory 83, a bank used corresponding to the predictiveinformation is set. From the bank, a predictive coefficient set fordoubling the number of pixels in the horizontal direction is read frompredictive coefficient sets stored at an address corresponding to theclass supplied from the classifying circuit 72. The predictivecoefficient set that is read from the bank is supplied to the predictivecalculating circuit 82.

The predictive calculating circuit 82 calculates Expression (1) with thepredictive coefficient set supplied from the coefficient memory 83 andpixel values of SD pixels that compose predictive taps supplied from thepredictive tap extracting circuit 81 and treats the predictive value ofthe considered pixel as a pixel value of a pixel of the intermediatepicture. The intermediate picture composed of the predicted pixels issupplied to the selector 22.

Since the learning process according to the other processing method inthe process mode (1) is basically the same as that of the dataprocessing apparatus 1 in the process mode (1), the description thereofwill be omitted.

Since the predictive process of the classifying adaptive processingportion 51-1 of the output data processing portion 24 in the example ofthe other processing method in the process mode (1) is basically thesame as that of the classifying adaptive processing portion 31-1 of heinput data processing portion 21 of the process mode (1), the detaileddescription thereof will be omitted. The classifying portion 61 of theclassifying adaptive processing portion 51-1 and the predictive tapextracting circuit 81 and the coefficient memory 83 of the adaptiveprocessing portion 62 extract class taps and predictive tapscorresponding to tap extraction information, predictive tap information,and coefficient information required in the example of the otherprocessing method in the process mode (1) and read a predictivecoefficient set from the coefficient memory 83. Corresponding to theresults, pixels for doubling the number of pixels in the verticaldirection are predicted.

In addition, since the learning process of the classifying adaptiveprocessing portion 51-1 of the output data processing portion 24according to the other processing method in the process mode (1) isbasically the same as the learning process of the classifying adaptiveprocessing portion 31-1 in the above-described process mode (1), thedescription thereof will be omitted.

In the above description, the classifying adaptive processing portion51-1 of the output data processing portion 24 executes the classifyingadaptive process for generating a picture whose number of pixels in thevertical direction is doubled so as to generate an HD picture. However,the present invention is not limited to such an example. For example, aswith the above-described process mode (1), according to the otherprocessing method in the process mode (1), a classifying adaptiveprocess for predicting pixels arranged on a line (line A) of which aline of an SD picture is present at the same positions as pixels thatcompose an HD picture and another classifying adaptive process forpredicting pixels arranged on a line (line B) of which a line of an SDpicture is not present at the same positions as pixels that compose anHD picture can be independently executed. In this case, theseclassifying adaptive processes of two classifying adaptive processingportions (for example, the classifying adaptive processing portions 51-1and 51-2) of the output data processing portion 24 can be executed.

In the above-described classifying adaptive process is described inInternational Patent Laid-Open No. WO 96/07987 and the correspondingU.S. Pat. No. 5,903,481 filed by the applicant of the present invention.

FIG. 16 shows an example of the structure of the data processingapparatus 1 that executes a process for generating a picture dedicatedfor a right eye and a picture dedicated for a left eye in theabove-described process mode (2). In the process mode (2), in thepost-processing portion 14, the multiplexing portion 141C is used.

In the example, the data processing apparatus 1 generates a picturededicated for a left eye (hereinafter referred to as left-eye picture)and a picture dedicated for a right eye (hereinafter referred to asright-eye picture) with SD picture data as input data supplied from anexternal device and outputs picture data (hereinafter referred to asstereo picture data) of the multiplexed picture (hereinafter referred toas stereo picture) as output data to an external device.

In this case, it is assumed that the SD picture data as input data is atelevision broadcast signal that has been A/D converted with a samplingclock of 13.5 MHz. In other words, the size of the SD picture is around720 pixels (horizontal)×480 lines (vertical) per frame.

In this example, a command that causes the process for generating aleft-eye picture and a right-eye picture to be executed is input to thefunction controlling portion 11. The function controlling portion 11outputs a control signal corresponding to the input command to the dataprocessing portion 12.

The function controlling portion 11 also extracts a vertical synchronoussignal and a horizontal synchronous signal from the television broadcastsignal and controls the data processing portion 12 and the multiplexingportion 141C of the post-processing portion 14 corresponding to thetimings thereof.

After the data processing portion 12 is initialized with a controlsignal supplied from the function controlling portion 11, the dataprocessing portion 12 generates a left-eye picture and a right-eyepicture with an SD picture that is input as input data.

The multiplexing portion 141C of the post-processing portion 14 disposeddownstream of the data processing portion 12 multiplexes the left-eyepicture data and the right-eye picture data supplied from the dataprocessing portion 12, generates stereo picture data, and outputs thegenerated stereo picture data as output data to an external device.

FIG. 17 shows an example of the connections of the data processingportion 12 in the process mode (2). In the example, in the input dataprocessing portion 21, the two classifying adaptive processing portions31-1 and 31-2 are used. SD picture data that is input as input data tothe data processing apparatus 1 is supplied to the classifying adaptiveprocessing portions 31-1 and 31-2.

The classifying adaptive processing portion 31-1 performs a classifyingadaptive process for generating left-eye picture data with the suppliedSD picture data. The classifying adaptive processing portion 31-1supplies the generated left-eye picture data to the selector 22. On theother hand, the classifying adaptive processing portion 31-2 performs aclassifying adaptive process for generating right-eye picture data withthe supplied SD picture data. The classifying adaptive processingportion 31-2 supplies the generated right-eye picture data to theselector 22.

The selector 22 supplies the left-eye picture data and the right-eyepicture data supplied from the classifying adaptive processing portions31-1 and 31-2 of the input data processing portion 21 to thepost-processing portion 14. In the post-processing portion 14, the routeof the selector 140 for the supplied left-eye picture data and right-eyepicture data is controlled corresponding to a control signal suppliedfrom the function controlling portion 11. The left-eye picture data andright-eye picture data are supplied to the multiplexing portion 141C.

In the process mode (2), the intermediate data processing portion 23 andthe output data processing portion 24 are not used.

Next, the predictive process of the classifying adaptive processingportion 31-1 of the input data processing portion 21 in the example willbe described.

The class tap extracting circuit 71 (see FIG. 8) of the classifyingadaptive processing portion 31-1 reads class tap extraction informationrequired in the example from the register 63A of the register group 63.The classifying adaptive processing portion 31-1 designates each ofpixels that compose the left-eye picture finally obtained in theclassifying adaptive process as a considered pixel. The left-eye picturedoes not actually exist before the predictive process is performed. Theclassifying adaptive processing portion 31-1 extracts class tapscorresponding to class tap extraction information corresponding to eachconsidered pixel and supplies the extracted class taps to-theclassifying circuit 72.

FIGS. 18A and 18B show examples of class tap extraction information inthe process mode (2). FIG. 18A shows class tap extraction informationfor a left-eye picture. FIG. 18B shows class tap extraction informationfor a right-eye picture. In FIGS. 18A and 18B, ● (black circle) and ◯(white circle) represent SD pixels. Now, it is assumed that in aleft-eye picture shown in FIG. 18A, a considered pixel is a blackcircle. In this case, class taps-are composed of 4 pixels×3 pixels thatcontains a considered pixel and that slightly deviate from theconsidered pixel (in the example, by 0.5 pixels). On the other hand,predictive taps (that will be described later) are composed of 7pixels×3 pixels that are wider to the right than the class taps by onepixel and to the left than the class taps by two pixels.

Unlike the left-side picture, in the right-side picture shown in FIG.18B, class taps are composed of 4 pixels×3 pixels that slightly deviatefrom a considered pixel (by 0.5 pixels as with the left-eye picture). Inthe right-side picture, predictive taps are composed of 7 pixels×3pixels that are wider to the left of the class taps by one pixel and tothe right of the class taps by two pixels.

The classifying circuit 72 reads class tap extraction informationrequired in the process mode (2) from the register 63A of the registergroup 63, decides the class of the considered pixel corresponding to theclass tap extraction information, and supplies the class to thecoefficient memory 83 of the adaptive processing portion 62.

The predictive tap extracting circuit 81 of the adaptive processingportion 62 reads predictive tap extraction information required in theprocess mode (2) from the register 63B of the register group 63. Thepredictive tap extracting circuit 81 extracts predictive taps from an SDpicture supplied to the adaptive processing portion 62 for theconsidered pixel corresponding to the predictive tap extractioninformation that has been read and supplies the extracted predictivetaps to the predictive calculating circuit 82. For example, aspractically shown in FIG. 19, predictive taps AR1 denoted by a dottedline are formed and supplied to the predictive calculating circuit 82.

The coefficient memory 83 reads coefficient information required in theexample from the register 63C of the register group 63 and sets a bankthat is used corresponding to the coefficient information. Thecoefficient memory 83 reads a predictive coefficient set for generatinga left-eye picture from predictive coefficient sets stored at an addressof the bank corresponding to the class supplied from the classifyingcircuit 72 and supplies the predictive coefficient set to the predictivecalculating circuit 82.

The predictive calculating circuit 82 calculates Expression (1) with thepredictive coefficient set supplied from the coefficient memory 83 andpixel values of the SD picture that compose the predictive taps suppliedfrom the predictive tap coefficient circuit 81 and designates thepredictive value of the considered pixel as the pixel value of a pixelthat composes the left-eye picture.

The left-eye picture composed of pixels having pixel values that havebeen predicted in such a manner are supplied to the selector 22.

Likewise, the classifying adaptive processing portion 31-2 processes aright-eye picture. Since the predictive process of the classifyingadaptive processing portion 31-2 is basically the same as the predictiveprocess of the classifying adaptive processing portion 31-1, thedetailed description thereof will be omitted. The classifying portion 61of the classifying adaptive processing portion 31-2, the predictive tapextracting circuit 81 of the adaptive processing portion 62, and thecoefficient memory 83 extract class taps corresponding to tap extractioninformation, predictive tap extraction information, and coefficientinformation required in the process mode (2) and read a predictivecoefficient set from the coefficient memory 83.

For example, the predictive tap extracting circuit 81 forms predictivetaps AR2 (denoted by a dot-dash line shown in FIG. 19) that deviate byseveral pixels to the right of the predictive taps AR1 formed by theclassifying adaptive processing portion 31-1. The amount of deviationbetween the predictive taps AR1 and AR2 can be set corresponding toparallax designated between a left-eye picture and a right-eye picturethat are generated.

In addition, the coefficient memory 83 of the classifying adaptiveprocessing portion 31-2 reads a predictive coefficient set forgenerating a right-eye picture from the bank and supplies the predictivecoefficient set to the predictive calculating circuit 82. The predictivecalculating circuit 82 calculates Expression (1) with the predictivecoefficient set supplied from the coefficient memory 83 and thepredictive taps supplied from the predictive tap extracting circuit 81and designates the predictive value of the considered pixel as the pixelvalue of a pixel that composes the right-eye picture.

The right-eye picture composed of pixels having predicted pixel valuesis supplied to the selector 22.

Next, the learning process in the process mode (2) will be described.When the learning process is performed by the classifying adaptiveprocessing portion 31-1, as shown in FIG. 20, an object is photographedby three television cameras (a left-eye camera L, a right-eye camera R,and a center camera C) disposed at positions having respectiveparallaxes. Among the three photographed pictures, pictures photographedby the left-eye camera L and the center camera C are used. However, inthe learning process of the classifying adaptive processing portion31-2, pictures photographed by the right-eye camera R and the centercamera C are used.

FIG. 21 shows an example of the structure of the classifying adaptiveprocessing portions 31-1 and 31-2 that perform a learning process in theprocess mode (2). In the structure, the thin-out circuit 91 of thestructure shown in FIG. 11 is not used.

A picture photographed by the center camera C is supplied to theclassifying portion 92 and the predictive tap extracting circuit 93. Apicture photographed by the left-eye camera L is supplied to the teacherdata extracting circuit 95. In other words, in the example, the picturephotographed by the center camera C is classified (namely, a class ofthe picture is decided). With teacher data that is a picturephotographed by the left-eye camera L, a predictive coefficient thatrepresents the correlation between the picture photographed by thecenter camera C and the picture photographed by the left-eye camera L isobtained for each class.

A set of predictive coefficients that have been obtained in such amanner is stored to the coefficient memory 83 of the classifyingadaptive processing portion 31-1.

Since the processes of the classifying portion 92 to the calculatingcircuit 97 are basically the same as those in the case shown in FIG. 11,the description thereof will be omitted.

The learning process of the classifying adaptive processing portion 31-2is the same as that of the classifying adaptive processing portion 31-1except that a picture photographed by the right-eye camera R is teacherdata. In other words, a predictive coefficient that represents thecorrelation between the picture photographed by the center camera C andthe picture photographed by the right-eye camera R is obtained. A set ofpredictive coefficients that have been obtained in such a manner isstored to the coefficient memory 83 of the classifying adaptiveprocessing portion 31-2.

The classifying adaptive process for generating a left-eye picture and aright-eye picture is described in Japanese Patent Laid-Open PublicationNo. HEI 9-55962 (laid open on Feb. 25, 1997) that was filed by theapplicant of the present invention.

FIG. 22 shows an example of the structure the data processing apparatus1 that executes a process for generating a luminance signal and colordifference signals in the process mode (3). In this example, an NTSC(National Television System Committee) format picture (hereinafterreferred to as NTSC picture) is supplied from an external device to thedata processing apparatus 1. In this example, it is assumed that theNTSC picture data supplied to the data processing apparatus 1 is only anSD picture as a so-called composite video signal of which a luminancesignal Y and color difference signals U and V are frequency-multiplexed(hereinafter, the composite video signal is referred to as compositesignal NTSC).

The data processing apparatus 1 separates the supplied NTSC picture datainto luminance Y and color differences U and V. When the NTSC picturedata is supplied to the data processing apparatus 1, the pre-processingportion 13 performs a gain compensating process, a subcarrierreproducing process, a phase shifting process, and so forth for the NTSCpicture data and supplies the processed NTSC picture data to the dataprocessing portion 12. The data processing portion 12 performs theclassifying adaptive process for the supplied data corresponding to thepre-learnt predictive coefficient set, generates the luminance signal Yand the color difference signals U and V, and outputs these signals.Hereinafter, these output signals are together referred to as componentsignal (Y, U, V). In the process mode (3), the pre-processing portion 13is not the above-described delaying circuit, but a circuit that performsthe gain compensating process, sub-carrier reproducing process, phaseshifting process, and so forth.

FIG. 23 shows an example of the connections of the data processingportion 12 in the process mode (3). In the process mode (3), in theinput data processing portion 21, the three classifying adaptiveprocessing portions 31-1, 31-2, and 31-3 are used. Signals NTSC-Y,NTSC-U, and NTSC-V of which the SD picture data that is input as inputdata to the data processing apparatus 1 is pre-processed in apredetermined manner (that will be described later) are supplied to theclassifying adaptive processing portions 31-1, 31-2, and 31-3,respectively.

The classifying adaptive processing portion 31-1 performs a classifyingadaptive process for generating the luminance signal Y with the suppliedsignal NTSC-Y and supplies the generated luminance signal Y to theselector 22. Likewise, the classifying adaptive processing portions 31-2and 31-3 perform classifying adaptive processes for generating the colordifference signals U and V with the supplied signals NTSC-U and NTSC-V,respectively, and supply the generated color difference signals U and Vto the selector 22.

The selector 22 supplies the luminance signal Y and the color differencesignals U and V supplied from the classifying adaptive processingportions 31-1, 31-2, and 31-3 of the input data processing portion 21 asoutput signals to the post-processing portion 14. The post-processingportion 14 outputs the supplied signals as they are.

In the process mode (2), the intermediate data processing portion 23 andthe output data processing portion 24 are not used.

FIG. 24 shows an example of the structure of the data processingapparatus 1 that accomplishes a Y/C separating device that performs aclassifying adaptive process for separating the composite signal NTSCinto the component signal (Y, U, V) in the process mode (3). As denotedby a dot line shown in FIG. 24, a block disposed upstream of classifyingcircuits 50, 51, and 52 corresponds to the pre-processing portion 13. Ablock disposed downstream of the classifying circuits 50, 51, and 52corresponds to the classifying adaptive processing portions 31-1, 31-2,and 31-3. A classifying portion 50 corresponds to the classifyingportion 61 of the classifying adaptive processing portion 31. Apredictive filter 53 corresponds to the adaptive processing portion 62of the classifying adaptive processing portion 31. In FIG. 24, theregister group 63 of the classifying adaptive processing portion 31 isomitted.

A composite signal NTSC is supplied from an input terminal 141. Asub-carrier reproducing circuit 142 extracts a sub-carrier from thesupplied composite signal NTSC and reproduces it. In addition, thesupplied composite signal NTSC is supplied to gain compensating circuits143, 144, and 145. The gain compensating circuit 143 performs a gaincompensation for converting the level of the composite signal NTSC intothe level of the luminance signal Y. The signal that is output from thegain compensating circuit 143 is the NTSC-Y signal and supplied to theclassifying circuit 150.

Likewise, the gain compensating circuit 144 compensates the gain. Amultiplying circuit 148 multiplies a sub-carrier that has beenphase-shifted by −123 degrees against the sub-carrier reproduced by thesub-carrier reproducing circuit 142 by the output of the gaincompensating circuit 144 and thereby generates the NTSC-U signal. TheNTSC-U signal is supplied to a classifying circuit 151.

Likewise, the gain compensating circuit 145 compensates the gain. Amultiplying circuit 149 multiplies a sub-carrier that has beenphase-shifted by −33 degrees against the sub-carrier reproduced by thesub-carrier reproducing circuit 142 by the output of the gaincompensating circuit 145 and thereby generates the NTSC-V signal. TheNTSC-V signal is supplied to a classifying circuit 152.

The classifying circuit 150 outputs class code P and data B generatedwith the supplied NTSC-Y signal to a predictive filter 153. Thepredictive filter 153 reads a filter coefficient corresponding to thesupplied class code P. The predictive filter 153 calculates the filtercoefficient and the supplied data B and thereby generates the luminancesignal Y. The generated luminance signal Y is obtained from an outputterminal 156.

The classifying circuit 151 outputs class code P and data B generatedwith the supplied NTSC-U signal to a predictive filter 154. Thepredictive filter 154 reads a filter coefficient corresponding to thesupplied class code P, calculates the filter coefficient and thesupplied data B, and thereby generates the color difference signal U.The generated color difference signal U is obtained from an outputterminal 157.

Likewise, the classifying circuit 152 outputs class code P and data Bgenerated with the supplied NTSC-V signal to a predictive filter 155.The color difference signal is obtained from the predictive filter 155through an output terminal 158. In such a manner, the component signal(Y, U, V) can be obtained.

Next, with reference to FIGS. 25 and 26, class taps and predictive tapsused in the process mode (3) will be described. As shown in FIGS. 25Aand 25B, when a considered pixel is pixel VO, class taps are composed ofpixel VO, pixels VA and VB on the upper and lower lines of pixel VO atthe corresponding positions of pixel VO in the field of pixel VO (0field), pixels VC and VD that are apart from pixel VO by one pixel inthe horizontal direction, and pixel VE of the preceding frame (−1frame=−2 frame) corresponding to pixel VO. The reason why pixels VC andVD that are apart from pixel VO by one pixel in the horizontal directionare used is to synchronize the phase.

On the other hand, as shown in FIG. 26, predictive taps are composed ofpixel VO, pixels that immediately surround pixel VO in the field ofpixel VO, the above-mentioned pixels VC and VD, and all thecorresponding pixels of the preceding frame of the field of pixel VO.Since a coefficient is obtained in consideration of the difference ofphases of pixels in the learning process (that will be described later),predictive taps can be extracted from pixels that have different phases.

Next, the learning process in the process mode (3) will be described.FIG. 27 shows an example of the structure of the classifying adaptiveprocessing portions 31-1, 31-2, and 31-3 that perform a learning processin the process mode (3). In the structure shown in FIG. 27, the thin-outcircuit 91 of the structure shown in FIG. 11 is not used. Instead, anNTSC encoder 191 is disposed. In FIG. 27, similar portions to those inFIG. 11 are denoted by similar reference numerals and the detaileddescription thereof will be omitted.

Predetermined luminance signal Y and color difference signals U and Vare supplied to the NTSC encoder 191. The NTSC encoder 191 encodes theinput signals into a composite signal NTSC. The composite signal NTSC issupplied to a classifying portion 92 and a predictive tap extractingcircuit 93. In other words, in such a structure, with the compositesignal NTSC, a considered pixel is classified and predictive taps areextracted. In addition, the luminance signal Y and color differencesignals U and V are supplied to a teacher data extracting circuit 95. Inthe structure, with teacher signal that are the luminance signal andcolor difference signals U and V, using the composite signal NTSC, apredictive coefficient set is generated.

A predictive coefficient set is generated for each of the luminancesignal Y and color difference signals U and V. The predictivecoefficient set corresponding to the luminance signal Y is applied forthe classifying adaptive processing portion 31-1 (namely, theclassifying circuit 150 and the predictive filter 153). The predictivecoefficient set corresponding to the color difference signal U isapplied for the classifying adaptive processing portion 31-2 (namely theclassifying circuit 151 and the predictive filter 154). The predictivecoefficient set corresponding to the color difference signal V isapplied for the classifying adaptive processing portion 31-3 (namely,the classifying circuit 152 and the predictive filter 155).

FIG. 28 shows an example of the structure of the data processingapparatus 1 that converts the aspect ratio in the above-describedprocess mode (4). In this example, the data processing portion 12increases the number of pixels only in the horizontal direction of apicture corresponding to picture data as input data (hereinafterreferred to as input picture) and changes the aspect ratio.

In this example, with three chronologically successive pixels (threepixels arranged successively in the horizontal direction), fourchronologically successive pixels (four pixels arranged successively inthe horizontal direction) of a picture (hereinafter referred to asoutput picture) corresponding to picture data as output data arepredicted. In this case, pixels of an output picture are newlypredictively created. In other words, in this example, an output pictureis generated in the state that the ratio of the number of pixels in thehorizontal direction of the input picture to the number of pixel in thehorizontal direction of the output picture is 3 to 4.

The function controlling portion 11 outputs a control signalcorresponding to an input command to the data processing portion 12. Thedata processing portion 12 selects a bank corresponding to the processmode (4) corresponding to the control signal supplied from the dataprocessing portion 12. The data processing portion 12 executes aclassifying adaptive process for generating four pixels with threepixels corresponding to a coefficient set of the selected bank. The dataprocessing portion 12 performs the classifying adaptive process for eachof the four pixels. Thus, the selector 140 of the post-processingportion 14 selects the multiplexing portion 141C that performs amultiplexing process corresponding to a control signal supplied from thefunction controlling portion 11.

The multiplexing portion 141C multiplexes picture data supplied from thedata processing portion 12. In other words, the multiplexing portion141C outputs an output picture of which the aspect ratio of the inputpicture was changed as output data to an external device.

FIG. 29 shows an example of the connections of the data processingportion 12 in the process mode (4). In this example, in the input dataprocessing portion 21, the four classifying adaptive processing portions31-1 to 31-4 are used. An input picture is supplied to the classifyingadaptive processing portions 31-1 to 31-4.

The input data processing portion 21 performs a predeterminedclassifying adaptive process for the input picture and outputs picturedata of which the ratio of the number of pixels in the horizontaldirection of an input picture to the number of pixels in the horizontaldirection of an output picture is 3 to 4 to the selector 22.

The selector 22 supplies picture data supplied form the input dataprocessing portion 21 to the post-processing portion 14. As wasdescribed above, in the post-processing portion 14, the selector 140selects the multiplexing portion 141C corresponding to a control signalsupplied form the function controlling portion 11. The picture datasupplied to the post-processing portion 14 is supplied to themultiplexing portion 141C. The multiplexing portion 141C multiplexes thepicture data supplied from the post-processing portion 14 and outputsthe multiplexed picture data as output picture data.

The intermediate data processing portion 23 and the output dataprocessing portion 24 are not used in the process mode (4).

Next, with reference to FIG. 30, the predictive process of theclassifying adaptive processing portion 31 of the input data processingportion 21 in the process mode (4) will be described. FIGS. 30A and 30Bschematically show pixels of an input picture and pixels of an outputpicture.

The classifying adaptive processing portion 31-1 performs a classifyingadaptive process in such a manner that a pixel of an output picture thathas a first phase relation to a pixel of an input picture (namely, apixel arranged on the same vertical line as a pixel of an input picture)is designated as a considered pixel and predicts the pixel value of theconsidered pixel. In the example shown in FIG. 30, pixels Pb1 and Pb5that compose an output picture and pixels Pa1 and Pa4 that compose aninput picture and that correspond to pixels pb1 and pb5 are used asconsidered pixels.

The classifying adaptive processing portion 31-2 performs a classifyingadaptive process in such a manner that a pixel that is immediatelychronologically preceded by a pixel whose pixel value has been predictedby the classifying adaptive processing portion 31-1 (namely, a pixelhaving a second phase relation to a pixel of an input picture) isdesignated as a considered pixel and predicts the pixel value of theconsidered pixel. In the example shown in FIG. 30B, pixel Pb1 is a pixelwhose pixel value is predicted by the classifying adaptive processingportion 31-1. Pixel Pb2 arranged adjacent to pixel Pb1 is a consideredpixel designated by the classifying adaptive processing portion 31-2.

The classifying adaptive processing portion 31-3 performs a classifyingadaptive process in such a manner that a pixel that-is immediatelychronologically preceded by a pixel whose pixel value is predicted bythe classifying adaptive processing portion 31-2 (namely, a pixel havinga third phase relation to a pixel of an input picture) is designated asa considered pixel and predicts the pixel value of the considered pixel.In the example shown in FIG. 30B, pixel Pb2 is a pixel whose pixel valueis predicted by the classifying adaptive processing portion 31-2. PixelPb3 arranged adjacent to pixel Pb2 is a considered pixel designated bythe classifying adaptive processing portion 31-3.

The classifying adaptive processing portion 31-4 performs a classifyingadaptive process in such a manner that a pixel (having a fourth phaserelation to a pixel of an input picture) that is immediatelychronologically preceded by a pixel (in the example shown in FIG. 30B,pixel Pb3) whose pixel value is predicted by the classifying adaptiveprocessing portion 31-3 is designated as a considered pixel and predictsthe pixel value of the considered pixel. In the example shown in FIG.30B, pixel Pb3 is a pixel whose pixel value is predicted by theclassifying adaptive processing portion 31-3. Pixel Pb4 arrangedadjacent to pixel Pb3 is a considered pixel designated by theclassifying adaptive processing portion 31-4.

In other words, since the number of pixels in the horizontal directionis increased in the ratio of 3 to 4, there are four types of phaserelations in the horizontal relation of a pixel (considered pixel) of anoutput picture and a pixel of an input picture. Each of the classifyingadaptive processing portions 31-1 to 31-4 performs a classifyingadaptive process corresponding to one of the four types of phaserelations and supplies the generated pixel to the selector 22.

The selector 22 supplies picture data (pixel) supplied from theclassifying adaptive processing portion 31 to the post-processingportion 14. As was described above, in the post-processing portion 14,the selector 14 supplies the supplied picture data to the multiplexingportion 141C corresponding to a control signal supplied from thefunction controlling portion 11. The multiplexing portion 141Cmultiplexes the picture data and outputs the multiplexed picture data toan external device. In such a manner, the aspect ratio is changed.

In the process mode (4), by increasing the number of pixels in thehorizontal scanning direction, the aspect ration is changed. Likewise,by decreasing the number of pixels in the horizontal scanning direction,the aspect ratio can be changed.

FIG. 31 shows an example of the structure of the data processingapparatus 1 that generates a picture having a different resolution inthe above-described process mode (5). In the example, the dataprocessing portion 12 generates picture data corresponding to aplurality of pictures having different resolutions with SD picture dataas input data supplied from an external device and outputs the generatedpicture data as output data to an external device.

FIG. 32 shows an example of the connections of the data processingportion 12 in the process mode (5). In the example, in the input dataprocessing portion 21, the classifying adaptive processing portions 31-1and 31-2 are used. SD picture data as input data is supplied to theclassifying adaptive processing portions 31-1 and 31-2.

The classifying adaptive processing portion 31 performs a classifyingadaptive process for supplied SD picture data and supplies the generatedpicture data having a first resolution to the selector 22.

The classifying adaptive processing portion 31-2 performs a classifyingadaptive process for the supplied SD picture data and supplies thegenerated picture data having a second resolution to the selector 22.

The selector 22 supplies picture data having the first resolutionsupplied from the classifying adaptive processing portion 31-1 of theinput data processing portion 21 to the classifying adaptive processingportions 51-1 and 51-2 of the output data processing portion 24.

The selector 22 supplies the picture data having the second resolutionsupplied from the classifying adaptive processing portion 31-2 of theinput data processing portion 21 to the classifying adaptive processingportions 51-3 and 51-4 of the output data processing portion 24.

In this example, in the output data processing portion 24, theclassifying adaptive processing portions 51-1 to 51-4 are used.

The classifying adaptive processing portion 51-1 performs a classifyingadaptive process for the picture data having the first resolutionsupplied through the selector 22 and generated by the classifyingadaptive processing portion 31-1 and outputs the generated picture datahaving the third resolution as output data.

The classifying adaptive processing portion 51-2 performs a classifyingadaptive process for the picture data having the first resolutionsupplied through the selector 22 and generated by the classifyingadaptive processing portion 31-1 and outputs the generated picture datahaving the fourth resolution as output data.

The classifying adaptive processing portion 51-3 performs a classifyingadaptive process for the picture data having the second resolutionsupplied through the selector 22 and generated by the classifyingadaptive processing portion 31-2 and outputs the generated picture datahaving the fifth generation as output data.

The classifying adaptive processing portion 51-4 performs a classifyingadaptive process for the picture data having the second resolutionsupplied through the selector 22 and generated by the classifyingadaptive processing portion 31-2 and outputs the generated picture datahaving the sixth resolution as output data.

In the process mode (5), a plurality of pictures having differentresolutions can be generated. Thus, for example, multi-windows havingdifferent sizes can be displayed.

The classifying adaptive process of each classifying adaptive processingportion of the input data processing portion 21 and the output dataprocessing portion 24 may be the same as that of the above-describeddata processing apparatus 1 in the process mode (1). In addition, in theprocess mode (5), the same class taps and predictive taps as those inthe above-described process mode (1) can be used.

FIG. 33 shows an example of the structure of the data processingapparatus 1 that converts the frame rate in the above-described processmode (6). In this case, the data processing apparatus 1 converts theframe rate of a picture that is input as input data.

The function controlling portion 11 supplies a control signalcorresponding to a command that is input in the process mode (6) to thedata processing portion 12. The data processing portion 12 selects abank for the process mode (6) corresponding to the control signalsupplied from the function controlling portion 11. The data processingportion 12 executes a classifying adaptive process for generating a newframe with picture data as input data corresponding to a coefficient setof the selected bank. In the process mode (6), the post-processingportion 14 selects the multiplexing portion 141C corresponding to thecontrol signal supplied from the function controlling portion 11 so asto multiplex a frame that is newly generated by the data processingportion 12.

A multiplexing portion 106 chronologically multiplexes picture data (asframes) supplied from the data processing portion 12 and outputs themultiplexed data as output data to the outside.

FIG. 34 shows an example of the connections of the data processingportion 12 in the process mode (6). In the example, in the input dataprocessing portion 21, the classifying adaptive processing portions 31-1and 31-2 are used. A picture as input data is input to each of theclassifying adaptive processing portions 31-1 and 31-2.

As shown in FIG. 35A, it is assumed that the input picture has a framestructure with a period of Ta. On the other hand, as shown in FIG. 35B,it is assumed that an output picture whose frame rate has been convertedhas a frame structure with a period of Ta/2. In other words, in thisexample, the frame rate is doubled.

In the classifying adaptive processing portion 31-1, the multiplexportion 141C performs a classifying adaptive process for generating aframe of an output picture that is chronologically preceded by a framethat composes an input picture. In reality, the classifying adaptiveprocessing portions 31-1 and 31-2 treats a frame in the unit of a pixel.In the classifying adaptive process of the classifyingadaptive-processing portion 31-1, a frame that composes an outputpicture is generated. In the example shown in FIG. 35, as shown in FIG.35B, frames B-2, B-4, and B-6 (white frames in FIG. 35B) of an outputpicture chronologically followed by frames A-1 to A-3 of an inputpicture shown in FIG. 35A are generated by the classifying adaptiveprocessing portion 31-1 (see FIG. 35B). The classifying adaptiveprocessing portion 31-1 supplies the generated frames to the selector22.

The-classifying adaptive processing portion 31-2 performs a classifyingadaptive process for generating a frame that composes an output pictureand that is chronologically followed by a frame that composes an inputpicture by the multiplexing portion 141C disposed downstream thereof. Inthe classifying adaptive process of the classifying adaptive processingportion 31-2, another frame that composes the output picture isgenerated. In the example shown in FIG. 35, as shown in FIG. 35B, framesB-1, B-3, and B-5 (hatched frames shown in FIG. 35B) of an outputpicture chronologically followed by frames A-1 to A-3 of an inputpicture shown in FIG. 35A are generated (see FIG. 35B). The classifyingadaptive processing portion 31-2 supplies the generated frames to theselector 22.

The selector 22 supplies the frames supplied from the classifyingadaptive processing portions 31-1 and 31-2 to the post-processingportion 14. The post-processing portion 14 supplies the frames selectedby the selector 140 corresponding to the control signal supplied fromthe function controlling portion 11 to the multiplexing portion 141C.The multiplexing portion 141C multiplexes the supplied frames in therule as was described with reference to FIG. 35B and outputs themultiplexed data as output data to an external device. In the processmode (6), in such a manner, the frame rate is converted.

FIG. 36 shows an example of class tap extraction information in theprocess mode (6). Next, the class tap extraction information will bedescribed with reference to FIG. 35A. In frame A-2 that contains aconsidered pixel, 3 pixels×3 pixels whose center pixel is the consideredpixel are extracted as class taps. In frames A-1 and A-3 that areimmediately preceded and followed by frame A-2, pixels corresponding tothose extracted as the class taps in frame A-2 are selected as classtaps. In other words, in the process mode (6), class taps are composedof 3 pixels×3 pixels×3 frames.

Predictive taps (not shown) include for example frame A-2 that has aconsidered pixel. A total of five frames composed of frame A-2, twoframes preceded by frame A-2, and two frames followed by frame A-2 areextracted. In each of the five frames, 5 pixels×5 pixels wider thanclass taps by one pixel are extracted. In other words, in the processmode (6), predictive taps are composed of for example 5 pixels×5pixels×5 frames.

The classifying adaptive process of the classifying adaptive processingportions 31-1 and 31-2 of the input data processing portion 21 may bethe same as that of the data processing apparatus 1 in the process mode(1).

In the above description, an example of which the present invention isapplied to picture data was explained. However, the present invention isnot limited to such an example. In other words, the present inventioncan be applied to other data such as audio data.

Moreover, in the above description, the data processing apparatus 1accomplishes the above-described process modes (1) to (6). However, thepresent invention is not limited to such an example. In other words, inthe data processing apparatus 1, with a predetermined predictivecoefficient set, a predetermined class tap structure, and apredetermined predictive tap structure, processes other than the processmodes (1) to (6) can be accomplished.

1. An information processing apparatus, comprising: a plurality ofclassifying adaptive processing circuits for performing a classifyingadaptive process for an input information signal; a switching circuitfor switching a connection relation among said plurality of classifyingadaptive processing circuits, wherein the information signals arepicture data composed of pixel information, wherein at least two of saidplurality of classifying adaptive processing circuits are configured forperforming the classifying adaptive process for the pixel informationhaving different phases and changing the number of pixel informationthat composes the picture data, wherein at least one of said pluralityof classifying adaptive processing circuit is configured for switchingthe configuration of extracted pixels as class taps or predictive tapsas the connection relation of said switching circuit is switched; and afunction controlling circuit to output a control signal to apre-processing circuit, a post-processing circuit and said switchingcircuit to control switching operations.
 2. The information processingapparatus as set forth in claim 1, wherein at least one of saidclassifying adaptive processing circuits is configured for switching thecorresponding classifying adaptive process for the correspondinginformation signal as the connection relation of said switching circuitis switched.
 3. The information processing apparatus as set forth inclaim 2, wherein at least one of said plurality of classifying adaptiveprocessing circuits is configured for switching a coefficient of thecorresponding classifying adaptive process so as to switch the processfor the corresponding information signal as the connection relation isswitched by said switching circuit.
 4. The information processingapparatus as set forth in claim 1, wherein the input information signalsare output through said plurality of classifying adaptive processingcircuits.
 5. The information processing apparatus as set forth in claim1, further comprising: a pre-processing circuit for performing apredetermined process for the input information signal and switching thepredetermined process as the connection relation is switched, wherein anoutput of said pre-processing circuit is input to the corresponding oneof said plurality of classifying adaptive processing circuits.
 6. Theinformation processing apparatus as set forth in claim 1, furthercomprising: a post-processing circuit for performing a predeterminedprocess for the corresponding input information signal and switching thepredetermined process as the connection relation is switched, wherein anoutput of one of said plurality of classifying adaptive circuits isinput to said post-processing circuit.
 7. The information processingapparatus as set forth in claim 1, wherein the information signals arepicture data composed of pixel information, and wherein one of saidplurality of classifying adaptive processing circuits is configured forperforming the classifying adaptive process based on the pixelinformation of the corresponding input information signal and predictingpixel information that has to be present between the pixel informationof the input information signal and pixel information adjacent theretoso as to improve the resolution of the picture data.
 8. An informationprocessing apparatus comprising: a plurality of classifying adaptiveprocessing circuits for performing a classifying adaptive process for aninput information signal; a switching circuit for switching a connectionrelation among said plurality of classifying adaptive processingcircuits, wherein the information signals are picture data composed ofpixel information, wherein at least two of said plurality of classifyingadaptive processing circuits are configured for performing theclassifying adaptive process and obtaining a plurality of picture datahaving different resolutions corresponding to the classifying adaptiveprocess performed by said plurality of classifying adaptive processingcircuits, wherein at least one of said plurality of classifying adaptiveprocessing circuit is configured for switching the configuration ofextracted pixels as class taps or predictive taps as the connectionrelation of said switching circuit is switched; and a functioncontrolling circuit to output a control signal to a pre-processingcircuit, a post-processing circuit and said switching circuit to controlswitching operations.
 9. The information processing apparatus as setforth in claim 8, wherein one of said plurality of classifying adaptiveprocessing circuits is configured for performing the classifyingadaptive process for the corresponding input information signal andobtaining picture data having a first resolution and another one of saidplurality of classifying adaptive processing circuits is configured forperforming the classifying adaptive process for picture data having thefirst resolution and obtaining picture data having a second resolution.10. The information processing apparatus as set forth in claim 8,wherein the information signals are picture data composed of pixelinformation and structured in the unit of a frame, and wherein one ofsaid plurality of classifying adaptive processing circuits is configuredfor performing the classifying adaptive process for the correspondinginformation signal that is input in the unit of a frame and generatingpicture data of frames chronologically preceded and followed by a frameof the input information signal.